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ABSTRACT 

Fourth in a series of six monographs on the use of 
new technologies in the instruction of learning disabled (LD) 
students, this paper explores issues related to the evaluation and 
selection of instruct ional software for LD students. Topics discussed 
include the following: (1) criteria for instructionally useful 
software (e.g., flexibility and unique display capabilities); (2) 
rationale for improved selection processes; (3) factors influencing 
selection (e.g., nature of documentation that accompanies the 
program); (4) producers of software; (5) evaluation forms and review 
procedur'as. A final section discusses elements of a comprehensive 
evaluation and selection instrument, including types of information 
to be gathered (vendor/program data, program description, 
instructional design features, and appropriate applications). (JW) 
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Introduction 



The market for educational software is very large and growing. Quality 
Educational Data, Inc., (Denver, CO) reports that in 1981-82, for example, 
there were 3^989 Apple computers In the schools, a year later there were 
15,773 and by 1983-84 there were 36,781 (Apple has 54% ol the market). 
Indeed, 75% of the schools in the country now have microcomputers. Where 
there is that kind of growth in computers there is automatically a powerful 
attraction for software development. By 1983-84 there were 1,947 separate 
companies who consid/^red themselves educational software publishers and some 
153 distributors of their materials. The volume of business connected with 
microcomputers has grown so rapidly and in such diverse ways that there are 
now 285 companies providing consultants and market research services. 

Given this remarkable growth and the certainty that it will continue in 
the foreseeable future, every effort should be made to open lines of coitf- 
munlcation between publishers and educational consxuners. This would allow 
the publishers to make their products known and the consxuners can be afforded 
the benefits that come from selecting high quality, Inst rue tionally targeted 
materials for use in their instructional programs. 

This paper is intended to serve as a baseline for Project CREATE 
(Center for Research in the Applications of Technology in Education), a 
baseline from which we can begin to develop prototypic selection and 
evaluation procedures to (1) help instructional staff to make effective 
choices among alternative materials, (2) encourage appropriate use of 
software in the curriculum so that their maximum cost-effectiveness is 
realized, and (3) provide feedback to publishers about features that they 
might want to incorporate as they develop new software for the educational 
market. 

CREATE* s efforts will involve an analysis of the "attractive" proper- 
ties o£ selected software and the development of evaluation forms that can 
be used to systematically identify the qualities that are important for 
learning disabled children. 
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1. Instmctlonal Usefulness of Software 



The instructional usefulness of software is generally defined by two 
criteria;^ (1) how we3.1 a program addresses the specific needs of students, 
and (2) how well a program fits into the school's curriculum. As simple as 
this statement appears./ only a small number of programs actually satisfy the 
definition. This is not to say that educational software that does meet the 
criteria for instructional usefulness cannot be "good" software. The term, 
good software, often is taken to mean that it is technically sound, easy to 
learn, or fun to use, but as an effective instructional tool it must be more 
than that. 

As suggested by Chambers and Sprecher (1983), a primary requirement for 
effective educational software is that it be developed by individuals who 
are knowledgeable in the subject matter content, computer technology, 
learning theory, and motivation of learners. In other words, the developer 
should understand and creatively plan for those variables that will 
(1) enable a teacher to reach a wide range of learners with varied needs, 
and (2) build into the program the flexibility necessary to correspond with 
different instructional objectives. CREATE has identified several features 
of creative programming that are desirable. These include: flexibility, 
ability to modify the program, exploitation of the computer's unique display 
capabilities, gaining and holding attention, real-time feedback, "patient" 
tutorial potential at?.d learner control. 

In addition to the instructional aspects of software, programs that 
permit recordkeeping, report generation, and profile building are valuable 
because of their usefulness for administrative monitoring of individualized 
instruction. 

The individvial needs of a student may vary from enrichment to reme- 
diation. For educational software to effectively address the special needs 
of handicapped learners, the software package must be flexible enough to 
construct "bridges" between various levels of learning abilities. Since a 
computer is capable of being programmed to provide such bridges, educators/ 
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programmers are able to utilize this tool in ways that complement more 
traditional methods of education. 

Flexibility . Software flexibility can be thought of as a means by 
which a program adjusts to the way an individual "best" learns. Among other 
things, flexibility includes adjustment of the rate of presentation, the 
size of the text, and the capacity to vork with alternative input devices. 
Learning disabled students, for example, may need to slow the rate of text 
presentation to effectively discriminate stimulus features such as letter 
shapes. Students with poor motor control may need to adjust the rate of 
presentation to match their response rate. Along with adjusting the rate 
of presentation, the capability of adjusting type size can be beneficial. 
In addition to being useful for the visually impaired, enlarging the type 
size would help to focus a learner on a specific aspect of a task. In this 
way, the computer display can function as a "place-marking card," where 
relevant stimuli are isolated from textual distractions in the display. 

Another important aspect of flexibility can be seen in those software 
programs that "branch" to a learner's ability level. Placement may ini- 
tially be arbitrary (as a "best guess") but is then adjusted on the basis of 
performance by the learner in comparison to pre-set criteria. The rationale 
for branching is that if a learner repeatedly gives incorrect responses, the 
program will branch to a less difficult level of instruction. Only when the 
student has mastered the content to criterion will the program branch to a 
more difficcJ.t level. 

Since most programs determine whether a student has mastered a level 
based on correct performance, the program is not directly assessing how a 
student is learning. Senf (1983) points out that performance-oriented 
programs are particularly limiting for learning disabled students because 
they do not determine why a student is giving incorrect responses. As Senf 
suggests, a student may be having difficulty with a certain reading task 
because he or she is "analyzing each word phonetically by syllable." 
Clearly, sending the student back to a less difficult level of instruction 
or review of the same material would not in itself teach the student to read 
efficiently. Instead, the program should assess incorrect responses by 



dentlfylng the underlying prerequisite skills, then branch the learner into 
L review or training of these skills. 

Suppose, for example, a program is designed to record error rates in 
reading of line copy, and it is deteriniined that most errors occur at the 
;ime of line change (where the eyes must return to the left margin and 
continue reading smoothly). The program might branch the learner into a 
training mode" where a window highlights the text to be read on a line- 
>y~line basis. The learner could then refine bJiS her vision skills, such 
LS coordination of foveal detail with peripheral awgirianess. In this example, 
.ndividualized instruction means that a teacher could set instructional 
parameters so that they are matched with the appropriate parameters for the 
.ndividual student. That is, a teacher could "tell" the program to branch 
:o the vision skills module under one set of conditions or to branch ahead 
Lf the criteria for mastery are satisfied by that particular student. 

Modification of the Program . The ability to modify a program allows a 
zeacher to tailor the instructional content. As just mentioned, this can be 
lecompllshed tlirough adjustment of the instructional parameters or by 
zeacher selection of the degree of learner control, or through a provision 
;or inserting or deleting information. 

The modification of program content is usually realized through the use 
>f authoring systems. An authoring system is a program that allows users to 
create individualized lessons without programming knowledge. For example, 
the program could provide a "user-friendly" way to design a quiz, present 
naterial for drill and practice, or create a vocabulary list in which words 
:an be added or deleted. Authoring systems vary in both sophistication and 
price. For example, a program such as "The Adaptable Skeleton" (Micro Power 
i Light Company, $34.95), allows users to create lessons within a multiple 
::hoice format. In contrast, a program such as "Assisted Instructional 
Development System" (Skillcorp Software, $395.00), includes such capa- 
bilities as branching at three levels, timed lessons, and a management 
system that allows generation of reports. 
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It Is becoming more common to see educational software companies 
Include "mini-authoring" systems with their programs (e,g., Advanced Ideas), 
The capacity for quick and easy Insertions Is Ideal for learning disabled 
students. A teacher could, for example, develop spelling lists based on 
particular letter patterns to facilitate word recognition as well as 
spelling practice. A benefit of using a mini-authoring system Is that 
program adjustments can be made that enable Individualized Instruction while 
maintaining the foundation of the program (I.e., the motivational properties 
of an Instructional game) . 

Unique Display Capabilities . There has been much criticism of the 
early use of computers as mere "page turners." Such criticism appears 
justified when considering the potential of using this medium In unique 
applications. Aside from the "real-world," only with a computer can 
students rotate a three-dimensional object or simulate driving a car. 
Unlike a static chalkboard or text, the computer can be programmed to be 
dynamic. 

To Illustrate, consider a hypothetical program where a student is 
taught to arrange a group of sentences into an expository paragraph. The 
objective would be to locate the sentence that contains the main Idea and 
organize the sentence components so that they relate to the main Idea (topic 
sentence) In proper sequence. 

After a learner completed a specified number of lessons, he or she 
would play a plnball game with the game balls earned during the lesson(s) . 
(The number of turns would be determined by the number of balls earned 
during the paragraph lesson.) 

Figure 1 Is an Illustration of how this program might appear on a 
computer display. The left side of the screen would be used for displaying 
the disorganized paragraph. The learner would select the sentences to build 
an organized paragraph on the right side of the screen. The number of game 
balls earned during a lesson would be displayed at the top of the screen. 
The plnball game, as Illustrated In Figure 2, would be designed to Improve 
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Figure 1 





! ' 


I like to eat them. 
Most pickles are green. 
When 1 do, my mouth puckers. 





Figure 2 
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eye-hand coordination by instructing the learner to use the "flippers" to 
direct the ball and keep it in motion/ The speed of the ball could be set 
by the learner. Let's see how this program might work: 

e The lesson would begin with the presentation of a main idea sentence 
and two or more supporting sentences displayed in random order. The learner 
would be instructed to bring the sentences (one at a time) to the organizer 
area and place them so that they relate properly to the main idea. 

• By using the cursor keys on the keyboard (or by using a joystick or 
graphics tablet) the learner would "pull" the sentences one-by-one into the 
paragraph organization area. 

(The simplicity in manipulating content makes the computer not only easy to 
use by individuals who have writing difficulty but it is exemplary for 
showing how components relate to a whole.) 

• The learner would earn a game ball (for play after the lesson) for 
choosing the proper sequence for each paragraph. 

• Incorrect responses would cause the program to "coach" the learner 
by highlighting the critical phrase in the sentence, e.g., tastes sour), and 
asking the learner for answers to key questions (e.g., " What tastes sour?") 

• The instructional parameters would be flexible so that a teacher 
could adjust the maximum number of "coached" responses allowed to earn a 
game ball. 

The ability to manipulate and move objects provides a sense of control 
over the environment. Interacting with and commanding a computer is a 
powerful and entertaining experience. This is particularly true for 
children who feel little control over how well they learn. There have been 
many reports of handicapped learners improving their self-concept through 
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the ego-boost they get from mastering a computer (Hagen, 1984; Turkel & 
Podell, 1984; Loebl & Kantrov, 1984). 

Interest and Attention . To gain a learner's attention programs often 
use a beep or bell sound. While this may be an effective means for gaining 
attention, holding attention is more difficult. Using game show sounds that 
are meant to suggest, "This is only the beginning .... Stay tuned for 
more excitement," can provide a sense of anticipation. ("What will I get if 
I win?"). Programs that have mini-authoring systems could allow students to 
select their own graphics and compose their own music to accompany their 
lessons. It could be quite exciting to be rewarded with one^s own creation. 

The bright colors available on some computer systems are attention- 
catching and eye-pleasing. There is something satisfying about seeing a 
representation of a yellow flower, or the darkness of space, as we expect to 
see them. If a school has only monochrome displays, the graphics should be 
as uncluttered and distinctive as possible. 

As Jerome Bruner put it, "The work of children is play." Particularly 
effective is the use of animation in software for the elementairy grade 
levels. Animation is familiar to young children and it is associated with 
fun and entertainment. When a cartoon character appears on the computer 
screen and says "Let's spell a word!," it seems like fun. 

The quality and style of graphics may be a more critical concern for 
students at the secondary level. Primitive looking graphics can be 
insulting to young adults. Programs aimed at the secondary grade level 
should provide graphics that are age-appropriate if they are to gain and 
maintain attention. 

Real-time Feedback . Turkel and Podell (1984) In discussing the learn- 
ing environment of Logo, give an excellent description of the advantage of 
real-time feedback; 
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The immediate feedback of a design on the screen enabled students to 
modify their instructions to the computer. By making modifications, 
students could change what the turtle was doing; this control provided 
the motivation to continue with the activity (p. 261). 

Instructional simulations, which are often presented in the form of 
problem-solving software, are ideal for maintaining Logo-like feedback and 
control within a variety of structured instructional environments. That is, 
a student can make decisions based on variables that the student controls. 

For example, a student may be simulating the role of an American Indian 
who plans to sign a peace treaty with a representative of the United States 
government. In traveling to the treaty-signing, he or she could encounter a 
range of barriers necessitating various decisions. These decisions lead to 
consequences, some of which could impact on the meeting. Cause and effect 
relationships become immediately apparent. If a strategy proves ineffective 
the student might call for help to find out why. Mistakes can be explained 
as soon as they occur and before they are repeated. Similarly, successes 
are rewarded immediately to ensure that they will be repeated. 

One of the real strengths of educational software is its potential for 
various feedback modes. This strength is especially realized when the rein- 
forcement schedules are consistent with learning theory and the rewards are 
interesting and different enough to motivate and hold attention. 

To illustrate how a program might use a variety of forms of feedback, 
considet^ a hypothetical math program. In this example, the student plays 
the part of a "math detective" who must find the solutions to a series of 
math problems in order to earn honor, success, and computer-money (points 
that add up in the "bank"). The learner moves through a series of rooms 
where, by correctly following certain mathematical operations, "new doors" 
are opened until the final goal is reached. (See Figure 3) 



Figure 3 



( — ^ 

To Open Door 
SoWe Problem : 



MMMiiMi 2 4- 2 + 2 ss ? 




A possible sequence is as follows: 

• The game begins with the student/detective receiving an 
assignment: "Solve the mysterious case of the missing — " 
(e.g., cat, ring). The learner would then be instructed to 
search the rooms for the missing item. In order to enter a 
new room the learner must correctly solve a math problem. 

• The student would receive randomly generated feedback (e.g., 
Good Job!; That's Right— Great) and graphics (a cat's toy 
might appear on one of the steps) as rewards for completing 
steps correctly. 

• The first two incorrect responses at each step might be met 
with a hint (such as highlighting the step that resulted in 
the incorrect solution) and a message to "try it again." 
After the next unsuccessful try, the problem could be 
completed step-by-step (by the program) along with the 
message, "Now you try it." 
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o All of the math operations would be highlighted and checked 
number-by-number so that the student could see where they 
were making an error. 

o Further errors might indicate that the student needs work on 
number relations, activating branch for special instruction 
. . . "How to be a math detective." 

• The difficult problem would appear again later in the 
session. 

• After correctly completing a certain segment of the math 
problems, the student would be rewarded (intermittently) 
with both music and graphics. 

By pressing a designated key a learner could receive help or clues at 
any time. The type of math problems and time allowed to complete them would 
be determined by the student. More earned points would be allotted to 
shorter solution times and harder problems. 

Handicapped learners need immediate feedback that is both non-judgmental 
and encouraging. As pointed out by Torgeson (1977), the learning disabled 
are typically passive in their approach to learning because they have 
received little or no payoff . The computer has tremendous potential for 
motivating and helping handicapped learners feel in control of learning. A 
characteristic of "good" software is that it allows a student to "attack" a 
lesson without fear of failure by offering plenty of hints and cues to mini- 
mize frustration. 

Learner control. Selecting an activity is one way students can assess 
their own strengths and weaknesses. It is important for students to dis- 
cover that "I'm good at this" and "I don't like that task." A well designed 
program will motivate students to try more difficult problems without fear 
of failure. A motivated learner may be determined to repeat an activity 
until his or her own criteria for success are met. "I'm going to win this 
time"; "Next time I won't type in a tp' because I know the right answer 
is a 'b.'" 
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Success for learning disabled students success may be, in part, knowing 
that learning itself can be controlled. A student may know, for example, 
that ^hen text moves across the screen at a slower rate, he or she can solve 
th& problem correctly. 

Educational management * In addition to instruction, programs can 
perform tasks that relate to instructional planning. Computers are 
excellent for accumulating information. Programs that update current 
records of students' strengths and weaknesses can build a profile for 
diagnostic purposes or aid a teacher in planning individualized instruction. 

Daily and cumulative reports are helpful to both the teacher and the 
student. A teacher can look at daily performance data and track a student's 
progress over time. For learning disabled students, the charting of skill 
development is one way of demonstrating that learning is actually taking 
place. 

Identifying software that if^ '^od" is not easy. Some "showy" features 
of the software may dominate the child's or teacher's impression of it, but 
other educationally important features may be missing. To help In selecting 
software that is cost effective, procedures must be used that are systematic 
and objective. 

2. Rationale for Improved Selection Processes 

Today's schools are faced with difficult choices about how to spend 
their resources most effectively to achieve quality education. Computers 
represent an exciting but potentially expensive investment for them, and the 
potential for waste in expenditures is a real one. For example, the rapidity 
with which the computer field is changing means that some machines will 
become obsolete and be replaced by faster and more versatile machines. What 
happens to the investment that has been made in the software for the old 
machine? Is it still usable on the new machine? Clearly, the estimated 
"life" of materials can be significant to the wise use of the school's 
limited financial resources. 
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A similar case can be made for investing in software that can be used 
in more than one way. Obviously, if it does "double duty" the.n its value is 
marginally greater, even when its expected "life" is relatively short. One 
example would be the adaptation of "regular" instructional software that has 
the potential to be used effectively with the handicapped. Perhaps a program 
for teacliing spelling can also be used to teach eye-hand coordination in a 
special needs student. Or a program that is designed to teach math opera- 
tions (adding or subtracting four-digit numbers) could be used to teach 
scanning by rows and columns to an individual who has trouble with hori- 
zontal and vertical spatial relationships. 

Another way in which maximxim cost-benefits can be realized is to 
prioritize on the acquisition of software in areas that teachers feel are 
critical to remediation of learners' problems or rapid advancement in a 
subject matter. That is, if it can be demonstrated that there is a poten- 
tial for "breakthrough," as might be possible by involving an autistic child 
in a particular learning game, then it is hard to deny the value of the 
material. Certainly, frequent teacher needs and learner needs must be given 
extra weight in the selection process. 

As school districts begin computer-supported instruction there is quite 
often a nucleus of dedicated and well-informed individuals who lead the 
movement, or, if it is a small school, the impetus can come from one person. 
Selection practices in this early adoption stage often tend to reflect the 
interests and biases of the individual or individuals involved. Thus some 
subject matter areas will be relatively "rich" in programs while others will 
be "poor." Similarly, there is sometimes an emphasis on one aspect of 
computer use, such as creative expression with LOGO, which reflects the 
perspective of the individual who is nominally in charge. 

Fortunately, there is increasing recognition that school policy needs 
to be established to guide the selection process. The policy should not be 
cumbersome, overly restrictive, or time consuming to Implement, but should 
be set forth, in writing, in such a way that teachers arc encouraged to 
become involved. One method for encouraging teacher Involvement is to have 
them participate in setting the policy. Another la to provide a stipend in 
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the summer for a team of teachers to engage in systematic review and 
evaluation of materials that can enhance thci program iu the coming school 
year. 

According to the Software Merchandising Directory, December 1983 
(Eastman Publishing, Encino, CA), the top ten "best sellers" in the 
educational marketplace are: 



Bank Street Writer 
Bumble Plot 

Pacemaker 

Gertrude's Puzzles 
Master Type 
Snooper Troops I 

Snooper Troops II 

Story Machine 
Type Attack 
Typing Tutor II 



Word Processing 

Learning Program - 
Number Pair Frequency 
Through Plotting 

Elementary Programming 
Through Games 

Shape/Color Patterns 

Typing Skills 

Detective-Type Games 
Library Skills 
Maps, Vocab 

Reasoning 
Info Classification 
Note Taking 

Language Arts-Writing 
Sentence, Paragraphs 

Typing 
Arcade Approach 

Typing 
Monitors Keyboard 



Br Oder bund 

The Learning Company 
Spinnaker 

The Learning Company 

Scarborough/Lightning 

Spinnaker 

Spinnaker 

Spinnaker 

Sirius 

Microsoft 



Even a casual inspection of this list shows the high priority being 
given in the schools to programs that (a) call for word use and expression, 
(b) have game-like characteristics, and (c) try to encourage thinking. 

3. Factors that Influence Selection 



Realistically, the most important limiting factor in selectior, is the 
school budget. Only a small percentage of the school district's operational 



expenses can be devoted to the purchase of software, so the Implication is 
clear — either restrict purchases to the very best and mo3t needed items, 
supplement the acquisition budget through fund-raising efforts of one kind 
or another, or seek out public domain materials. Materials in the public 
domain can be elusive, however, because even though some may be quite well 
done, their authors have no marketing budget and knowledge of their avail- 
ability usually depends on word-of-mouth or bulletins circulated by 
"cooperative" user groups. 

When high quality materials are identified, there is a natural 
inclination to want more than one copy so thr,c the program can be used 
simultaneously in more -than one class or school. This brings up another 
problem. One copy may be affordable, but multiple copies may not. It 
should be noted that copyright law does apply to author's and publisher's 
rights in the case of software. Piracy, while known to* exist, is not a 
viable option for the schools. Special purchase agreements can take the 
need for multiple use into account, either through discounts on additional 
copies or through negotiated rights to distribute the software through a 
local area network (LAN) from a central location. 

Another factor that influences the selection and utilization of 
software is the nature of the documentation that accompanies the program. 
Is the software "ready to go" or is there a need for considerable study by 
teachers to implement it smoothly and effectively? If this question is not 
addressed by the buyer before purchase, the material may never reach its 
full potential for use in the school. It is disappointing but true that 
poorly written documentation has kept some otherwise useful software from 
receiving the attention it deserves. To reduce dependence on accompanying 
materials that may be of questionable quality, there is a trend toward 
making the program itself "teach" the user through "help" options. Of 
course, there is a limit to which this can be done without using up the 
available memory in the machine. As time goes on, newly emerging hardware 
with much greater stOui?.;^^ capacity and software that is integrated will 
counterbalance the et^^^i^ jtement that users be taught through separate print 
materials. 
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An obvious factor, and at least partly a function of the Initial 
district decisions, limits selection to the range of software that will 
operate on the school's machines. This is not an insignificant factor, for 
unlike a book that can be substituted when a new edition comes out, the 
program must be compatible. While most of the very popular software is 
available for more than one type of computer, there are still many fine 
programs that can only run on equipment that the school doesn't possess — or 
exists only in a school other than the one in which the software is needed. 
It is the availability of hardware that has caused such a strong domination 
of the school software market by Apple. Recent promotional policies by IBM 
(particularly at the higher education level) are challenging that domination. 

Assuming compatibility and the establishment of priority needs, a 
factor that should not be underestimated is the importance of critically 
appraising the quality pf instructional design in a given program. Whether 
it is intended as a central part of the curriculum, such as teaching touch- 
typing in a typewriting class, or as adjunct material, such as the use of 
"brain buster" puzzles to occupy students productively during eaimed "free 
time," there is a need to judge whether the program will hold student 
Interest and if it is technically correct. Regardless of whether evaluation 
is teacher-based, committee-based, or student-based, it should involve 
systematic evaluation . More will be said about this in a subsequent section 
of this paper. 

4. Producer Variables and Sources for Software 

Teachers and schools have not yet developed a high set of standards 
demanding polish in microcomputer .software in the same way they have for 
textbooks. While printed materials that are commercially acquired must be 
"accepted" by review committees using criteria that are tough for publishers 
to comply with, software still enjoys a kind of forgiveness and an over- 
looking of deficiencies among many districts. As the market fills, however, 
the reliability of the publisher in using high standards during product 
development will become much more important. 
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The tremendous amount of variation In the quality of education software 
Is paralleled by variation In the size and experience of the producing 
organizations. That Is not to say that "bigness" of the firm Is necessarily 
correlated with the "goodness" of the program, but rather that If the 
development Investment Is large enough, the purchaser may more reasonably 
assume a greater amount of Internal and external review to find "bugs" prior 
to publication than Is usually the case for an Individual operating 
Indep endent ly • 

Major developer-publishers of software Include divisions of traditional 
educational publishers such as Addlson-Wesley, Sllver-Burdett, and Scott 
Foresman. With some exception, the strategy that Is favored by these large 
firms Is to develop a block of curriculum materials that are demonstrably 
correlated with the scope and sequence of "mainstream" Instruction In the 
language arts and math areas, and more recently In social studies and 
science. These materials are aimed at the majority of school chj.ldren who 
are "on track" In their progress through the grades and course work. Apart 
from several curriculum adaptation projects supported by the Office of 
Special Education Programs, U. S. Department of Education, these large 
publishers are not concentrating on the special education market, for this 
thin market Is not able to support the magnitude of Investment that they 
make In royalties, duplication, packaging, marketing. Including brochures, 
"free" examination copies, and convention demonstrations. In any case; the 
focus of Instruction In these large scale curriculum centered programs Is 
typically on academic skills and not on underlying special needs such as 
training In perceptual processes. 

At the next level of magnitude are the publishers who specialize In 
supplementary materials. Some prominent examples are Sunburst Coimnunl- 
cations, DLM, Hartley, Spinnaker, and the Learning Company. The strategy 
here Is not to develop a complete curriculum In microcomputer format, but 
rather to offer a potpourri of titles that will appeal to a variety of 
teachers In a school district. There are also a large number of vendors of 
software who handle the products from different software publishers. 
Examples* Include K-12 Mlcromedla and Opportunities for Learning, Inc. These 
firms are distributors, not publishers, and apart from exercising some 
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selectivity about the items they choose to carry, do not directly affect the 
introduction of new products for the marketplace. 



Some developer-publisher firms are striving to fill a particular niche 
in the market. Koala^ for example, emphasizes software that can be used 
with the touchpad that it manufactures. A very few are exploring the use of 
microcomputers and laser videodisc technology to develop interactive tech- 
nology at a high level of instructional sophistication. An example is 
MicroEd, Inc. , a midwest firm that is headed by Ihorwald Esbensen, a long- 
time educator and advocate of individualized learning. In an unusual 
marketing strategy, MicroEd has made available a number of software grants 
to school districts in which up to $10,000 worth of its educational products 
are donated to set up elementary school computer-assisted libraries. 

Additionally, the development of educational software is by no means 
restricted to well-funded publishers of instructional materials, but also 
includes a large number of "cottage industry" types, people who generate 
creative ideas and follow through in developing them. Unfortunately, they 
often give insufficient thought to incorporating features of design that 
lead to successfully marketed products. 

Yet another level of software production is by teachers themselves. As 
indicated previously, some teachers work independently, then share their 
products through a coordinating center or agency which emphasizes public 
domain software. A prime example of the cooperative producer-user approach 
is SOFISWAP, a joint project of the San Mateo (CA) County Office of 
Education and Computer Using Educators (CUE). Some 6,000 educators are 
involved and programs are available for Apple, Atari, IBM, Commodore, and 
other, hardware. 

Other teachers may be part of a network of interlocking schools and 
administrative units such as the Minnesota Educational Computing Consortium 
(MECC). There are differences in these two approaches, of course, with one 
of the most important being the opportunity to pilot and field test materials 
within a consortium of schools before they are marketed nationally. The 
MECC strategy has worked so successfully that it can be thought of as an 
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"educator's publishing house" rather than a commercial publishing house. 
Illustrative of the MECC type of product is "Elementary Volume 7," a nine 
disk package for non-readers and beginning readers (see the review of this 
package in the Journal of Learning Disabilities > May 1984) • 

Lastly, one must consider the students and faculty within one's own 
district as an untapped resource for authoring new software. Ovar the long 
term it is likely that "friendly" authoring systems that continue to be 
developed will encourage teachers and students to create instructional 
materials just as they now prepare quizzes, handouts, and exercises for 
homework. (There is debate on this point, however. Some writers seem to 
feel that teachers' time will always be so limited that authoring will be 
unlikely. We take the view that as the systems become more and more 
user-friendly, more and more people will be tempted to participate.) 

The preceding discussion of sources has been concerned with the 
educational software marketplace In general. The special education 
marketplace represents a fraction of the larger market, of course, and is 
concentrated in two areas. On the one hand, there are clearly a variety of 
programs to help in the I£P process. On the other hand, much of what is 
being evaltiated and reviewed by teachers of the learning disabled* are 
really not materials targeted specifically at the learning disabled popu- 
lation but rather at the general school population. For example, drill and 
practice of academic content used for special children may not be "special" 
at all. There are very few developers who are preparing software uniquely 
designed to overcome or ameliorate specific learning disabilities. 

Car twright (1984) in his article entitled "Computer Applications in 
Special Education", cites a study by Chaing at the Cupertino Union School 
District in California. The purpose of the study was to determine the 
effectiveness of computer assisted instruction with mildly handicapped 
students. Throughout the course of the study teachers created 975 lessons 




*In the Journal of Learning Disabilities > for example, the "LD Scan" reviews 
are contributed to by panels of teachers all reviewing the same product. 
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In reading, math and language arts. The teachers then tried out the 
materials with 200 handicapped, elementary-level children. Comparison with 
a control group Indicated that local level teacher-lnltlated development of 
programs could be effective. 

Walker and Hess (1984) suggest that software producers field test an 
early version of a program (under realistic field conditions), where they 
can observe Its use and Identify unforeseen problems. Similarly, Singer 
(1983) recommends field testing as a way to Improve the quality of 
software. Singer suggests that both teachers and students use separate 
evaluation forms to provide feedback about a program.* This formative 
evaluation strategy is designed to compare teachers* and students* 
perceptions of a program* s strengths and weaknesses. 

Truett (1984) conducted a survey of educational software producers 
across the country to ascertain the extent to which they used field 
testing. Her findings must be viewed with caution since there was an 
extremely low rate of return to the questionnaire (56 returned of 406 
mailed). Keeping in mind the limitations on representativeness that the low 
return implies (for example, the data probably overstate the real 
situation) , it is nevertheless interesting to examine some aspects of the 
survey results. Of those who responded: 

• 69% of the producers did some type of field testing 

• 51% involved six teachers or less 

• 44% used local schools 

• 46% involved fewer than 50 students 

• 37% relied on teacher evaluations 

• 32% Indicated that an In-house review took place 

Truett properly acknowledged that there was a strong possibility that the 
non-respondents were biased toward no field testing. She has requested 
funding to investigate this possibility. If one assumes that the data are 
meaningful, then it is immediately apparent that the extent of field testing 
is quite small given the number of copies that may be sold and the number of 



*See The Computing Teacher , 1983, 11(3), page 36, for the Singer Student 
Software Questionnaire and Singer Teacher Software checklist. 
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users (teachers and students) who would be affected over time. Considering 
this limitation^ It Is most Important that school districts use care In the 
acquisition of software. 

Given the diversity of sources for materials and such unevenness In 
their products In terms of (a) their specificity with regard to the needs of 
the learning disabled, (b) their "solldness" In the degree to which they are 
Integrated with the curriculum* and (d) their merit as examples of the art 
of programming, It Is obvious that evaluation and selection procedures 
should be In place In every school district that elects to serve Its LD 
population of students via microcomputer. The next section will discuss 
Important aspects of evaluation and selection as they are currently being 
practiced and should be practiced In the schools. 

5. Evaluation Forms and Review Procedures 

Table 1 Is a comparison of evaluation forms used by different software 
review organizations. This table Is not meant to represent an exhaustive 
survey, but rather the Intent Is to Illustrate the major elements considered 
In present evaluation and selection practices. 

The table shows that most evaluation Instruments concentrate on a 
description of vendor/ program data, an evaluation of a program's clarity 
and usefulness, and the Instructional design features of a program. These 
elements are primarily concerned with the general effectlvenesa of a 
program. That Is, how well the program "performs." 

The "appropriate applications" comparison In the table Is concerned 
with the Identification of the underlying skills necessary for effective 
Information processing. A program may teach these skills directly 
(expressed as the educational objective) or Indirectly (such as the visual 
tracking that Is Involved In playing many video games). Unfortunately, 
appropriate applications data Is almost entirely absent from most edu- 
cational evaluation forms. Consequently, teachers who are looking for 
software that could be adapted for use with special populations can only 
surmise whether the software Is "right" for teaching basic, underlying 
skills. 
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RQCoramended by Council for Except lonel Children 

In order to select software that can adapt to the needs and ability 
levels of a wide range of learners, the evaluation instrument must have some 
provision for addressing the key question: "How can this piece of software 
be used effectively in special education?" 



Many commercially available programs have the potential to be used 
effectively with the LD population. For example, a program could be adapted 
for use with special learners simply by having teachers develop introductory 
activities to help students practice the skills necessary for mastering the 
software content. Clearly, special education teachers should be involved in 
the evaluation of "regular" software along with regular teachers so that 
they can identify skill areas in the software that may be used effectively 
with special learners. 
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Ideally, school evaluation and selection procedures should take into 
account field test results supplied by developers and narrative descriptions 
from experienced computer-using educaf^ors. Reviews from third party sources 
(e.g., TECC, EPIE) could also provide schools with a perspective that 
facilitates the selection process. 

Below are descriptions of two commercial publications that can help in 
the development of an evaluation/ selection process at the school district 
level. 

The Software Catalog; Microcomputers . Published by Elsevier 
Science Publishing, Inc., (52 Vanderbilt Ave., New York, NY 
10017), the Software Catalog is a 794 page reference catalog of 
educational software. The catalog contains general vendor/program 
data. Two unique categories are; (1) date of release (useful for 
keeping track of programs under development), and (2) source code 
availability. (The source code is available when a program has 
been written in a high-level language (i.e., Pascal). The source 
code enables users to modify parts of a program.) 

The Software Catalog is published every six months and is continu- 
ously updated with new software information (updated th' :»e months 
after each publication.) 

Hively^s Choice; School Year 1983-8/ This comprehensive volume 
has been edited and co-published by Wells Hively. The volume is 
available through' Hively* s Choice Publications, Inc., and The 
Continental Press, Inc., Elizabe^^Atown, Pennsylvania 17022. 

Hively 's Choice is a collection of selected quality software that 
has been reviewed and listed by instructional objectives, Instruc- 
. tional ideas and examples (based on field testing), curriculum 

connections, and preparation time. Hively' s Choice proposes to be 
a great time-saver for teachers anf' parents who are looking for 
; quality programs that perhaps meet a particular Instructional 

objective. Subsequent volumes are planned for each year. 

. ■> ■ 
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The following Is a short description of those commercial organizations 
that were presented in the previous table: 



School Mlcroware Reviews > published by Dresden Associates (P.O. Box 
246, Dresden, Maine 04342). School Mlcroware Reviews Is a peri- 
odical of educational software evaluations by three or more 
reviewer/educators. Each Issue Includes a narrative evaluation of 
at least 50 software packages — two to five programs In each major 
academic area. The publication Is available three times a year by 
subscription or through a public access database called Resources 
In Computer Education (RICE). 

EPIE (Educational Products Information Exchange) . In 1982, the 
EPIE Institute joined with Consumer's Union of the U.S. to evaluate 
both hardware and software products. EPIE has provided educators 
with extensive training In using their evaluation form. The form 
uses both a numerical rating scale and descriptive sentences, and 
It Is 12 pages in length. 

Once certified by EPIE as a "Courseware Analyst," the educator 
provides EPIE and Consumer *s Union with software evaluation 
reports. The reports are published in "Microcomputer Courseware 
PRO/ FILES" and "Microcomputer Hardware PRO/FILES" and are available 
to schools on a subscription basis, (see following subsection 
"on-line databases" for the scheduled on-line implementation.) For 
further information write to: EPIE Institute, 1018 Keith Avenue, 
Berkeley, CA 94708 

Software Reports . Published by Allenbach Industries, (2101 Las 
Palmas Drive, Carlsbad, CA 92008), Software Reports is a 
compilation of almost 400 educational software programs that have 
been evaluated by an "independent software review board" called The 
Evaluation Committee (TEC). The programs have been divided into 14 
different subject areas. There is a separate section for special 
education. 
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Each program was evaluated by five criteria: (1) documentation, 
(2) ease of use, (3) program content, (4) instructional technique, 
and (5) educational usefulness. Programs are listed by subject 
category. An overall evaluation "grade" (A, B, C, D) is given to 
each program. Subscribers can receive future updates at a reduced 
cost by completing a survey questionnaire. 

MicroSIFT . Housed at the Northwest Regional Educational 
Laboratory, in Portland, Oregon, MicroSIFT is a clearinghouse for 
educational software. The software evaluations are performed by 
the MicroSIFT Dissemination Nercv^vTrk, SIFTnet, a group of over 200 
educational agencies located across the nation. 

The MicroSIFT evaluation instrument is comprised of two forms: 
(1) the "Courseware Description," which consists of a two-page 
checklist and short narratives of Vendor/Program Data, and (2) the 
"Courseware Evaluation" form, which includes a two-page evaluation 
using a 4-point rating scale (SA=strongly agree; SD=strongly 
disagree) and short narratives. 

The software reports are the composite of the evaluations made by 
two teachers and a "computer education specialist." These software 
reviews are made available to educators through the MicroSIFT 
Dissemination Network and the on-line database, Resources in 
Computer Education (ELLCE). 

On-line Databases 

Producing objective software evaluations and disseminating them by 
means of print-media is costly to the publisher, and therefore to the 
buyer. Since most review publications do not accept advertising to maintain 
impartiality, they must rely on subscriptions for financial support. 
However, with individual subscriptions ranging from approximately $50 to 
$2G0 a year, many schools cannot afford them. A case in point was the 
periodical "Courseware Report Card," which recently ceased publication due 
to financial losses (see Educational Technology , May 1984). 
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The critical issue is: "How can published software evaluations be 
disseminated so that schools can afford to use them in their evaluation/ 
selection procedures?". Many print-media periodicals are also offering 
on-line databases. These databases provide access to extensive searching 
capabilities for a wide range of information about educational software. 

Below are descriptions of two on-line educational software databases: 

RICE . A product of the Northwest Regional Educational Laboratory, 
the RICE database has been available through Bibliographic 
Retrieval Services (BRS) since 1982. RICE offers users a software 
database with 2400 records which can be searched by computer type, 
producer, grade level, subject, or application. A free text 
searching option allows users to locate character strings imbedded 
within the record, such as descriptive phrases. 

EPIE On-line . Soon to be released through CompuServe is an 
educational software database called "EPIE on-line". This database 
will provide on-line access to approximately 5000 software programs 
listed in The Educational Software Selector (TESS). (TESS is avail- 
able in print form from Teachers* College Press.) Information can 
be searched by type of computer, subject, grade, title, use in 
school (regular, remedial, special education, and administration), 
price, supplier; record keeping, network, review, and EPIE review. 
Information is delimited by using successive individual search 
characteristics from the above list. 

In summary, in order to select software that can adapt to the needs and 
ability levels of a wide range of learners, the evaluation instrument must 
have some provision for addressing the question: "How can this piece of 
software be used effectively in special education?" 

The key to informed evaluation and selection practices lies in the 
development of review standards that are both systematic and comprehensive. 
The next section will discuss in greater detail the elements of a com- 
prehensive evaluation instrument and how it might be implemented in the 
schools. 
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6. Elements of a Comprehensive Evaluation and b'electlon Instrument 

The voltxme of software being produced and the tremendous amount of 
overlap that exists between Information channels i^bout educational software 
makes It obvious that no teacher can stay abreaat of what is currently avail- 
able or know whether It is of high quality or not. Even If the largest part 
of the teacher's day was not already devoted to the teaching process, the 
task would be Insurmountable. Consequently, teacher judgments must often be 
made on the strength of what someone else says about software that Is 
available and worthwhile. 

What "someone else" should the teacher trust? Who has the time, capa- 
city and required knowledge to undertake the "right" kind of evaluation — 
evaluation that Is meaningful and thorough? 

For the sake of Illustrating the problem and Its solution, suppose 
three teachers agreed to pool their evaluative reviews of software. One of 
the first things they would have to do would be to agree on what they should 
look for and how they would judge It. They would. In effect, need to estab- 
lish criteria that cotild be applied with consistency so that each person has 
a basis for judging the merits of an evaluation conducted by the other(s.). 

Assuming that our three teachers are then able to enlist the help of a 
fourth Individual who may not be skilled In the subject matter but never- 
theless Is Interested In education and willing to help, they would probably 
ask that person to take on those parts of the evaluation activity that do 
not demand an educational background such as the gathering of Information 
about unit cost or the identification of the source and type of 
microcomputer on which the program would run. 

To carry our example one step farther, let us imagine that each of the 
cooperating teachers has special skills in an area that the others lack or 
in which they aren* t particularly Interested. For example, one teacher 
might be keenly Interested in perception as it relates to the act of 
reading. That teacher would naturally want to make direct, personal 
judgments about the potential of the* software to help reach particular 
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perception-training objectives. A second teacher might concentrate on 
evaluations of software for comprehension-building and the third on 
spelling. Presumably, the software identified by the three teachers could 
be used in different, but complementary, ways. 

In essence, then, it is clear that (a) different persons can be use- 
fully engaged in the review process, (b) if the results of their efforts are 
to be interpreted reliably their evaluative procedures should be uniform, 
and (c) the same software can be used in more than one way. 

The present state-of-the-art in evaluation and selection of software is 
definitely not uniform. Moreover, it seems likely that comprehensive evalu- 
ations will be precluded unless complementary roles are adopted during the 
process of review. Unless important changes are made in practice, the role 
that chance plays (e.g., what salesman comes to call, or what conversations 
occur at a professional meeting) will continue to play a major role in what 
software a teacher selects. 

We believe that a generalizable evaluation/ selection form and protocol 
is needed that (1) distinguishes the types of information that need to be 
gathered for appropriate educational decisions about applications, 
(2) permits that information to be gathered by a team of people at different 
times, and (3) can be used by teachers to identify important program 
features. A prototype version of an AIR/CREAT£ Software Evaluation form is 
attached to this paper. 

Types of information to be gathered 

We envision four major types of information about software that should 
be collected: 

1. Vendor/Program Data 

2. Prograsi Description 

3. Instructional Design Features 

4. Appropriate Applications 
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1. Vendor/ Program Data 



The following three categories of information do not require profes- 
sional judgment or expertise but rather are based primarily on information 
supplied by the vendor. Accordingly, they can comprise an introductory 
section in the evaluation form. This information can be filled out by cleri 
cal staff, classroom aides, or volunteers in advance of critical review by 
faculty. 

A. Regardless of whether a product i& eventually selected or not, 
a record should be maintained of the product name, the source 
from which it is obtained, and cost-per-unit information. 

B. Information needs to be recorded that clearly identifies the 
brands/models of equipment: that the software can be run on as 
well as memory requirements or peripheral equipment needs. 

C. Vendor literature often provides sufficient information to 
identify the instructional purpose of the material, tha^ is, 
whether it is for regular instruction, for remediation, or for 
enrichment and general interest. The age/grade level (primary, 
intermediate, secondary, postsecondary) should be evident from 
the promotional literature as should the overall nature of the 
program (drill and practice, game, testing, authoring, etc.) 

2. Program Description 

The next sections of the evaluation can be completed by an assigned 
review committee within the district or across districts. These would 
contribute toward building a basic, descriptive file that individual 
teachers could turn to for initial indications of program quality. 

A. . A synopsis of content should be >rcitten. This should entail 

a brief description, together with strong points and weak 
points. 

B. Although local evaluation is always important, it is often 
helpful to note the evaluations by others that might have 
appeared in professional journals or in publications that 
specialize in software reviews. 

C. If the producer (developer-publisher) has made field testing 
data available, this should be noted. However, if the 
testing is questionable or claims seem exaggerated, only 
limited importance should be attached to it. 
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If the software has been tried within the district this 
should be noted as well as the extent of testing and the 
results obtained. 

3. Instructional Design Features 

Nexty a section of the evaluation form should address instructional 
design features. These are central to the educational value of the 
pro($ram. Without question, they will be important to any final judgment 
abc ' the appropriateness of instructional material to the particular needs 
of '^.ach learning disabled child. One evaluative criterion that should not 
be overlooked is the quality of the documentation and its completeness as a 
teacher guide. To the extent that the documentation gives clear directions 
about how the software fits into the curriculum, indicates instructional 
objectives, provides pre- and post-measures, and suggests supplemental 
activities, the teacher can see ways in which it can be integrated into the 
Instructional program. When these elements are missing it is harder to 
integrate the software in a timely and effective way. (Indeed, when 
students discern that software is "off topic," they tend to view it as an 
escape from class rather than part of an overall program.) 

Consider, for example, a child whose frustration tolerance is low, who 
is having problems in mastering a specific learning task. Is the software 
sequential in nature? Is provision made in the program for branching to 
remedial information or is the same item played back until the right guess 
is' made (with little, if any, change in the learner* s understanding of why 
the answer was correct)? Similarly, with the short-attention-span child, 
does the program allow the- learner to enter and exit specific parts of the 
program without laboriously going back through the whole sequence or waiting 
until the program is complete before stopping? 

A. The completeness of the educational experience involves such 
elements as its relevance to the curriculum, statements of 
purpose, worksheets and related activities suggested, 
accuracy and correctness of information, and records of 
performance. 

B. User friendliness is always important, but for the handi- 
capped student, simplicity may be crucial. Instructions 
must be clear and readable, it must be easy to get started, 
the program should not "crash," and graphics, color, and 
sound should be used in a way that assists the learner. 



C. The program should respond to individual differences. 
Ideally, it should motivate, be adaptable, adjust to ability 
levels, and give appropriate "hints.** 

D. Under learner control, the program should allow selection of 
activities and easy access as well as the setting of appro- 
priate rates and help options. 

E. Learners should be given positive reinforcement and kept 
informed of their progress. Explanations should be given so 
that errors are not repeated. 

Notes may be required on software that is non traditional in 
its design properties, such as unstructured, creative 
programs in which input is neither right nor wrong, but 
simply incremental. 

4. Appropriate Applications 

In order to maximize the value of the evaluation-selection procedure to 
teachers, it is necessary to think in terms of how teachers would prefer to 
initiate contacts with administrators or specialists when they are seeking 
leads to software. When teachers use software with their special students, 
they should have in mind a specific goal for each learner. Teachers might 
well ask, /'I'm interested in some material that might help with Johnny's 
letter reversal problem. What do you have to suggest?" This implies the 
need for a. section of the evaluation form that takes into account the 
variety of instructional goals that teachers may ask about. - 

The goal may be improvement in basic vision skills, cognitive 
processing skills, perceptual enabling skills, or academic performance 
skills. .It is important, therefore, that each software item be evaluated in 
terms of its utility in one or more of these goal areas. 

A. A much overlooked area of instruction that can be especially 
important to some learning disabled children is vision 
skills training. Vision skills that lend themselves to 
training by computer are neuromuscular in nature, such as 
binocular fusion, accommodation, and tracking and fixation. 
Vision skills do not refer to lens-correctable conditions, 
to surgical correction, or to degenerative pathologies. 
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B. A type of instruction that is fundamental, but often 
neglected, encompasses those enabling skills that frequently 
cause learning disabled students to perceive their environ- 
ment in a faulty or inefficient way. These include such 
underlying skills as being able to distinguish meaningful 
cues on the basis of features or patterns, establish later- 
ality and orientation in space, recognize part-whole rela- 
tionships, and attend, decode, and classify perceived 
stimuli. Even though much of the software that is currently 
in the schools may not address these learning goals as their 
primary purpose, there is every possibility that teachers of 
the learning disabled will be able to use some software 
creatively to achieve these secondary purposes. 

C. At a higher level, software can be effective in instruction 
aimed at developing cognitive processes. These are the 
intellectual skills that are essential to memory processes, 
analytic processes, development of meaning and context in 
comprehension, and general thinking involving reasoning and 
problem solving. Until these skills are developed, any 
learning of academic subject matter is likely to be labori- 
ous and fraught with errors. Although some software exists 
that addresses cognitive processes directly, sometimes 
software in the subject matter areas can be used to build 
these skills. 

D. Most educational software calls for performance of academic 
skills such as reading, writing, math, science, social 
studies, and vocational and business subject matter. Other 
programs deal with art, foreign language, social skills, and 
computer literacy. More precisely, programs are often aimed 
at some specific aspect of a subject matter; in math this 
might be fact acquisitioa, number relations, math operations, 
problem solving, logic, or other topics. Again, secondary 
uses of the software may be very worthwhile and need to be 
identified during the evaluation process. 

Building an evaluation database 

It should be clear that during the review process it may be recognized 
that the same software can be used in a variety of ways— the clever teacher 
and the creative special education department will take advantage of this 
opportunity to maximize the instructional return on their dollars. 

For example, in the premier issue of Computers, Reading and Language 
Arts, Kuchinskas (1983) suggests 22 ways in which microcomputers can be used 
in a reading/language arts curriculum. Spanning instruction, testing, and 
data storage, she identifies the following applications: 
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1. 


Drill and Practice 


12. 


Objectives Production 


2. 


Tutoring 


13. 


Inventories 


3. 


Assessment 


14. 


Word Processing 


4. 


Record Keeping 


15. 


Data Banks 


5. 


Prescz'intions 


16. 


Videodiscs 


6.* 


Inters Stive Language Programs 


17. 


Simulations 


7. 


Readability 


18. 


Computer Programming 


8. 


Language Analysis 


19. 


Staff Development 


9. 


CLOZE Passage Generation 


20. 


Action Research 


10. 


Vocabulary List Generation 


21. 


Computer Literacy 


11. 


lest Item Generation 


22. 


Other 



District staff caa often anticipate some of the likely questions 
teachers may pose about available software. Use of the evaluation form will 
help to identify which software does in fact deal with those kinds of 
instructional focus. However, there remains a problem of readily storing 
the information so that it can be accessed at the time the teacher needs 
it. If the amount of information to be stored were not so large (a variety 
of software that could be used for specific applications) it would be 
tempting to simply ask a knowledgeable district-level supervisor to recall 
the information. That approach is not really feasible given the great 
amount of software available and the numerous variations that are possible 
for its use. 

Suppose, for example, that the teacher wants software to help improve a 
learner's attention span. The computer could be used as a database^ 
allowing teachers to make inquiries (through a keyword system) so that 
relevant software possibilities identified with the descriptor "attention 
span" would be identified. These eligible pieces of software could be 
delimited further according to age/appropriateness, or computer-brand by 
simply imposing additional terms as search constraints. 

By using a file management program, a dlatrlct-level team could define 
a database using a keyword system. The evaluation Instrument should serve 
as the foundation for selection of kcyr^/ords. As each piece of software is 



evaluated it would be entered into the database and "flagged" by keywords 
(e.g., attention; discrimination; problem solving) as checked on the 
evaluation form. 

The advantage of implementing a local level database is that a teacher 
could efficiently locate software that: (1) meets the search criteria the 
teacher establishes, (2) is adequate according to a comprehensive evaluation 
procedure, and (3) is currently available within the district. 

Summary 

During the initial evaluation and review of any particular piece of 
software, it is unlikely that any one person will be able to make the 
"final" judgments about all of the educational applications that are fea- 
sible. One strategy would be to use a committee approach to the review of 
the material so that its possible value can be evaluated separately for 
(a) vision skill training, (b) perceptual enabling skills training, 
(c) cognitive processing skills training, and (d) academic performance 
skills training. Whether the applications aspect of the software evaluation 
is carried out by an individual or a committee, however, the intent should 
be to identify a variety of ways in which the particular software item can 
be used, not just the purpose as stated by the vendor. The goal should be 
to think of the material as a resource that, if added to the district's 
collection, should be used to the fullest to assure the best dollar return 
on investment. 

By developing and using a form that provides a checklist approach to 
the listing of appropriate applications, information about the software can 
be readily encoded into a database. The database could then be accessed by 
teachers and specialists in answer to the question, "What do you have that 
will help me teach "— ?" . 

Staff in CR£ATE have developed a preliminary version of an evaluation 
and selection form for use in the schools (it is attached). It will be 
.field tested in the 1984-85 school year in cooperating schools around the 
, country. ^ 
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EDUCATIONAL SOFTWARE EVALUATION FORM 



AIR/CREATE 



Copyright 
1984 
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I. vendor/program data 



A. . PRODUCT AND SOURCE DESCRIPTION 

1. Program Name 

2. Distributor Name ' 

3. Address 



4. 
5. 
6. 
7. 
8. 



Telephone ( ) 



Single Program 

Cost $ 

Terms for updated versions 
Back-up Policy 
Includes back-up copy_^ 



Part of a Series 



Terms for multiple copies 

Hard disk version available 

B. HARDWARE REQUIREMENTS 

1. Microcomputer (Enter model) 

Ap pi e A t ar i Commo d or e_ 



IBM 



Other 



2. Memory Required 



3. 



4. 



Storage Medium 

Diskette Tape cassette ROM cartridge 

Equipment (Enter R for required, 0 for optional) 

Printer^ Joystick 

Disk drive 1 Mouse 
Disk drive 2 



Color monitor 

Graphics tablet 



Voice recognition device_ 

Voice synthesizer 

Other 



5. Optional Devices 

Joystick Mouse^ 

Graphics tablet 

EDUCATIONAL OBJECTIVE 
1. 



^ Voice synthesizer^ 
Other 



2. 



3. 



Instructional Purpose 
Remediation Regular Instruction Enrichment 

Suggested Grade/Ability Level 

Primar y Intermediate Secondary Postsecondary 



Presentation Mode 
Drill and practice^ 

Educational game 

Problem solvin^^ 

Slmulation^^ 
Tutorial 



Information retrieval^ 

Testing 

Authoring system 

Classroom management_ 
Game (for fun only) 
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II. PROGRAM DESCRIPTION 



A. SYNOPSIS OF CONTENT 
1. Descriptio n 



2. Strong po;::its_ 



3. Weak points 



B, EXTERNATiY REVIEWED 

1. Source 

2. Rating; 

3. Cautions . 



C. FIELD TESTED BY PiJODUCER 

1. Results are available 

2. Number of students involved 

3. Claims made 



D. LOCAL TRYOUT 

1. Who/where tried 

2. Number of studants involved 

3. Results 
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III. INSTRUCTIONAL DESIGN FEATURES 

A. EDUCATIONAL COMPLETENESS 

1. Integrated with curriculum 

2. Names prerequisite skills 

3. States instructional outcomes clearly 
4» Suggests related activities 

Before using computer 
After using computer 

5. Information presented is accurate/correct 

6. Included worksheets 

7. Provides for testing 
Pretest 

Posttest 

8. ^ Provides performance records 

Based on sound learning theory 

B. USER FRIENDLINESS 

1. Operating manual is clear 

2. On-screen instructions are easy 

3. On-screen text is readable 

4. Menu driven (learner can begin easily) 

5. Safeguarded against "crashes" 

6. Graphics and color assist rather than distract 

7. Sound can be turned off /on 

C. LEARNER SENSITIVITY 

1. Learner motivation is sustained 

2. Program can be adapted to individual needs 

3. Automatically adjusts to student *s ability level 

4. Gives clues, hints to encourage correct answer 
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D. LEARNER CONTROL 

1. Can control rate of presentation 

2. Can select activity (without repeating 
completed segments) 

3. Can repeat any activity 

4. Can exit at any time 

5. Can get help on-line at any time 

E . FEEDBACK/REINFORCEMENT 

1. Positive reinforcement is immediate 

2. Incorrect answers are explained 

3. Remedial loops are built in 

4. Students are rewarded only for correct responses 

5. Student is informed of success rate 



IV. APPROPRIATE APPLICATIONS 



A. VISION SKILLS INVOLVING NEUROMUSCULAR FUNCTIONING 

1. Binocular fixation (coordination of the two eyes) 

2. Accommodation (focusing a clear image) 

3. Tracking (appropriate eye movements and fixation) 

B. ENABLING SKILLS INVOLVING PERCEPTION 

1. Discrimination 

a. of features 

b. of patterns 

2. Attention 

a. span 

b. concentration 

c • alertness 

3. Classification 

a. groupin g 

b. figure/ground 

4. Cue Identification 

a. searching 

b. differentiation 

5. Coordination and Integration 

a. visual/mo tor^ 

b. auditory/motor 

c • visual/ audit ory 

6. Part-whole Relationships i 

a. verbal 

b. graphic ^ 

7. Orientation 

a. lateral 

b. spatial 

8. Decoding of Letter Shapes 

a. typed or printed 

b. written 

c. reversals/inversion s 

9. Decoding of Words 

a. letter sets 

b. word lengt h 

c. word shap es( asc enders / desc enders ) 
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C. PROCESSING SKILLS INVOLVING COGNITION 



1. Analytic Techniques 

a. In reading (e.g. , key ideas, sentence structure) 

b. In math (e.g., conservation, relative magnitude) 

c. In science (e.g., cause/effect, hypothesis checking) 

d. In social studies (e.g., timelines, political interactions) 

2. Understanding and Meaning 

a. vocabulary and definitions 

b. use of context 

c. comprehension 

3. Recall of Information 

a. chunking and grouping 

b. mnemonic techniques 

c. rehearsal 

4. General Purpose Thinking 

a. reasoning 

b. problem solving 

c. study strategies 

D. PERFORMANCE SKILLS INVOLVING SUBJECT MATTER 

1 . Reading 

a • speed 

b. whole word recognition 

c. phonics and blends 

d . comprehension 

e. other_ 

2. Writing and Composition 



a. 


spelling 


b. 


punctuation 


c. 


capitalization 


d. 


word usage 


e. 


parts of speech 


f . 


organization of ideas 




syntax 


h. 


proofreading 


i. 


other 


Math 


a. 


fact acquisition 


b. 


number relations 


c. 


math operations 


d. 


problem solving 


e. 


logic 


f. 


graphing 


8- 


other 



4 • Science 

a. fact acquisition 

b. observing and measuring 

c. data interpretation 

d. formulating generalizations and hypotheses 

e . problem solving; 

f . othe r 

5. Social Studies 

a. fact acquisition 

b. data interpretation 

c. identification of concc^pts and issues 

d. evaluation of evidence 

e. formulating generalizations and hypotheses 

f . other • ■ 



6 • Vocational/Business 

a. technical skills 

b. work readiness 

c. career awareness 

d . other 



7. Foreign Languages 

a . vocabulary 

b. rules and patterns 

c. comprehension 

d. other 



8. Arts 

a. perception of visual properties 

b. analysis and interpretation 

c. technique and eye-hand coordination 

d. creativity of design and form 

e. other 



9. Social Skills 

a. handling stress and conflict 

b. taking responsibility 

c. communicating effectively 

d. analyzing feelings 

e . group participation 

f . other 



10. Computer Literacy 

a. general computer use 

b . programming 

c. graphics 

d . other 



11. 



Other Subject Matter 

a. ' • • 



